Distilling Reliable Information From Unreliable Theories

Suppose a domain expert gives us a domain theory which is meant to classify examples as positive or negative examples of some concept. Now suppose, as is often the case, that the expert speciies parts of the theory which might be in need of repair, as opposed to those parts of the theory which are certainly not in need of repair. We say that such a theory is partially mutable. There might be some non-empty set of examples each of which has a classiication in the partially mutable theory which is invariant under all possible sets of repairs to unreliable components of the theory. We call such examples stable. We present an eecient algorithm for identifying stable examples for a large class of rst-order clausal theories with negation and recursion. We further show how to use stability to arbitrate between the theory and a noisy oracle to improve classiication accuracy. We present experimental results on some awed theories which illustrate the approach.